{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用大型语言模型作为评审者清理现有的偏好数据集\n",
    "\n",
    "_作者：[David Berenstein](https://huggingface.co/davidberenstein1957) 和 [Sara Han Díaz](https://huggingface.co/sdiazlor)_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **库**: [argilla](https://github.com/argilla-io/argilla), [hf-inference-endpoints](https://github.com/huggingface/huggingface_hub)\n",
    "- **组件**: [LoadDataFromDicts](https://distilabel.argilla.io/dev/components-gallery/steps/loaddatafromdicts/), [UltraFeedback](https://distilabel.argilla.io/latest/components-gallery/tasks/ultrafeedback/), [KeepColumns](https://distilabel.argilla.io/latest/components-gallery/steps/groupcolumns/), [PreferenceToArgilla](https://distilabel.argilla.io/latest/components-gallery/steps/textgenerationtoargilla/), [InferenceEndpointsLLM](https://distilabel.argilla.io/latest/components-gallery/llms/inferenceendpointsllm/), [GlobalStep](https://distilabel.argilla.io/latest/sections/how_to_guides/basic/step/global_step/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在本教程中，我们将使用 **distilabel** 清理数据集，利用大型语言模型（LLMs）作为评审者，通过提供 AI 反馈来评估数据的质量。[distilabel](https://github.com/argilla-io/distilabel) 是一个用于工程师的合成数据和 AI 反馈框架，帮助快速、可靠且可扩展地构建基于经过验证的研究论文的管道。查看文档 [这里](https://distilabel.argilla.io/latest/)。\n",
    "\n",
    "为了评估响应，我们将使用与 distilabel 集成的 [无服务器 HF 推理 API](https://huggingface.co/docs/api-inference/index)。该服务免费但有请求限制，允许你通过简单的 HTTP 请求测试和评估超过 150,000 个公开模型，或者使用你自己的私有模型，推理任务在 Hugging Face 共享基础设施上进行。如果需要更多计算能力，您可以使用 [Hugging Face 推理端点](https://huggingface.co/docs/inference-endpoints/guides/create_endpoint) 部署自己的推理端点。\n",
    "\n",
    "最后，为了进一步整理数据，我们将使用 [Argilla](https://github.com/argilla-io/argilla)，它允许我们对数据质量提供人工反馈。Argilla 是一个为 AI 工程师和领域专家提供的协作工具，帮助他们为项目构建高质量的数据集。查看文档 [这里](https://docs.argilla.io/latest/)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 开始"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 安装依赖\n",
    "\n",
    "为了完成本教程，你需要通过 pip 安装 distilabel SDK 和一些第三方库。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install \"distilabel[hf-inference-endpoints]\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install \"transformers~=4.0\" \"torch~=2.0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们进行必要的导入："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "from datasets import load_dataset\n",
    "\n",
    "from distilabel.llms import InferenceEndpointsLLM\n",
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import (\n",
    "    KeepColumns,\n",
    "    LoadDataFromDicts,\n",
    "    PreferenceToArgilla,\n",
    ")\n",
    "from distilabel.steps.tasks import UltraFeedback"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你需要一个 `HF_TOKEN` 才能使用 HF 推理端点。在此 Notebook 中直接登录以使用它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from huggingface_hub import login\n",
    "\n",
    "login(token=os.getenv(\"HF_TOKEN\"), add_to_git_credential=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### （可选）部署 Argilla\n",
    "\n",
    "你可以跳过此步骤，或者将其替换为任何其他数据评估工具，但如果缺乏数据质量，模型的性能将受到影响，因此我们建议你查看你的数据。如果你已经部署了 Argilla，可以跳过此步骤。否则，你可以按照 [此指南](https://docs.argilla.io/latest/getting_started/quickstart/) 快速部署 Argilla。\n",
    "\n",
    "同时，你需要将 Argilla 作为 distilabel 的附加组件安装。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install \"distilabel[argilla, hf-inference-endpoints]\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这种情况下，我们将清理一个偏好数据集，因此我们将使用 Hugging Face Hub 上的 [`Intel/orca_dpo_pairs`](https://huggingface.co/datasets/Intel/orca_dpo_pairs) 数据集。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<iframe\n",
    "  src=\"https://huggingface.co/datasets/Intel/orca_dpo_pairs/embed/viewer/default/train\"\n",
    "  frameborder=\"0\"\n",
    "  width=\"100%\"\n",
    "  height=\"560px\"\n",
    "></iframe>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = load_dataset(\"Intel/orca_dpo_pairs\", split=\"train[:20]\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们将打乱 `chosen` 和 `rejected` 列，以避免数据集中的任何偏差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def shuffle_and_track(chosen, rejected):\n",
    "    pair = [chosen, rejected]\n",
    "    random.shuffle(pair)\n",
    "    order = [\"chosen\" if x == chosen else \"rejected\" for x in pair]\n",
    "    return {\"generations\": pair, \"order\": order}\n",
    "\n",
    "dataset = dataset.map(lambda x: shuffle_and_track(x[\"chosen\"], x[\"rejected\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dataset.to_list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### （可选）创建自定义步骤\n",
    "\n",
    "步骤是 distilabel 管道中的一个模块，用于操作、生成或评估数据等任务。提供了一组预定义的步骤，但你也可以创建 [自定义步骤](https://distilabel.argilla.io/latest/sections/how_to_guides/basic/step/#defining-custom-steps)。与之前章节中的数据预处理不同，你可以使用自定义步骤来打乱列。这个步骤应该放在一个单独的模块中，以便导入并在管道中使用。在这种情况下，管道将首先使用 `LoadDataFromHub` 步骤加载 `orca_dpo_pairs` 数据集，然后应用 `ShuffleStep` 步骤。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \"shuffle_step.py\"\n",
    "from typing import TYPE_CHECKING, List\n",
    "from distilabel.steps import GlobalStep, StepInput\n",
    "\n",
    "if TYPE_CHECKING:\n",
    "    from distilabel.steps.typing import StepOutput\n",
    "    \n",
    "import random\n",
    "\n",
    "class ShuffleStep(GlobalStep):\n",
    "    @property\n",
    "    def inputs(self) -> List[str]:\n",
    "        return [\"instruction\", \"chosen\", \"rejected\"]\n",
    "\n",
    "    @property\n",
    "    def outputs(self) -> List[str]:\n",
    "        return [\"instruction\", \"generations\", \"order\"]\n",
    "\n",
    "    def process(self, inputs: StepInput) -> \"StepOutput\":\n",
    "        outputs = []\n",
    "\n",
    "        for input in inputs:\n",
    "            chosen = input[\"chosen\"]\n",
    "            rejected = input[\"rejected\"]\n",
    "            pair = [chosen, rejected]\n",
    "            random.shuffle(pair)\n",
    "            order = [\"chosen\" if x == chosen else \"rejected\" for x in pair]\n",
    "            \n",
    "            outputs.append({\"instruction\": input[\"instruction\"], \"generations\": pair, \"order\": order})\n",
    "\n",
    "        yield outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from shuffle_step import ShuffleStep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 定义管道"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了清理一个现有的偏好数据集，我们需要定义一个包含所有必要步骤的 `Pipeline`。类似的工作流也可以用于清理 SFT（监督微调）数据集。接下来，我们将详细讲解每个步骤。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 加载数据集\n",
    "我们将使用刚才打乱的数据集作为源数据。\n",
    "\n",
    "- 组件：`LoadDataFromDicts`\n",
    "- 输入列：`system`、`question`、`chosen`、`rejected`、`generations` 和 `order`，与加载的字典列表中的键相同。\n",
    "- 输出列：`system`、`instruction`、`chosen`、`rejected`、`generations` 和 `order`。我们将使用 `output_mappings` 来重命名列。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([{'system': '',\n",
       "   'question': \"You will be given a definition of a task first, then some input of the task.\\nThis task is about using the specified sentence and converting the sentence to Resource Description Framework (RDF) triplets of the form (subject, predicate object). The RDF triplets generated must be such that the triplets accurately capture the structure and semantics of the input sentence. The input is a sentence and the output is a list of triplets of the form [subject, predicate, object] that capture the relationships present in the sentence. When a sentence has more than 1 RDF triplet possible, the output must contain all of them.\\n\\nAFC Ajax (amateurs)'s ground is Sportpark De Toekomst where Ajax Youth Academy also play.\\nOutput:\",\n",
       "   'chosen': '[\\n  [\"AFC Ajax (amateurs)\", \"has ground\", \"Sportpark De Toekomst\"],\\n  [\"Ajax Youth Academy\", \"plays at\", \"Sportpark De Toekomst\"]\\n]',\n",
       "   'rejected': \" Sure, I'd be happy to help! Here are the RDF triplets for the input sentence:\\n\\n[AFC Ajax (amateurs), hasGround, Sportpark De Toekomst]\\n[Ajax Youth Academy, playsAt, Sportpark De Toekomst]\\n\\nExplanation:\\n\\n* AFC Ajax (amateurs) is the subject of the first triplet, and hasGround is the predicate that describes the relationship between AFC Ajax (amateurs) and Sportpark De Toekomst.\\n* Ajax Youth Academy is the subject of the second triplet, and playsAt is the predicate that describes the relationship between Ajax Youth Academy and Sportpark De Toekomst.\\n\\nNote that there may be other possible RDF triplets that could be derived from the input sentence, but the above triplets capture the main relationships present in the sentence.\",\n",
       "   'generations': [\" Sure, I'd be happy to help! Here are the RDF triplets for the input sentence:\\n\\n[AFC Ajax (amateurs), hasGround, Sportpark De Toekomst]\\n[Ajax Youth Academy, playsAt, Sportpark De Toekomst]\\n\\nExplanation:\\n\\n* AFC Ajax (amateurs) is the subject of the first triplet, and hasGround is the predicate that describes the relationship between AFC Ajax (amateurs) and Sportpark De Toekomst.\\n* Ajax Youth Academy is the subject of the second triplet, and playsAt is the predicate that describes the relationship between Ajax Youth Academy and Sportpark De Toekomst.\\n\\nNote that there may be other possible RDF triplets that could be derived from the input sentence, but the above triplets capture the main relationships present in the sentence.\",\n",
       "    '[\\n  [\"AFC Ajax (amateurs)\", \"has ground\", \"Sportpark De Toekomst\"],\\n  [\"Ajax Youth Academy\", \"plays at\", \"Sportpark De Toekomst\"]\\n]'],\n",
       "   'order': ['rejected', 'chosen']}],\n",
       " True)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_dataset = LoadDataFromDicts(\n",
    "    data=dataset[:1],\n",
    "    output_mappings={\"question\": \"instruction\"},\n",
    "    pipeline=Pipeline(name=\"showcase-pipeline\"),\n",
    ")\n",
    "load_dataset.load()\n",
    "next(load_dataset.process())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 评估响应\n",
    "\n",
    "为了评估响应的质量，我们将使用 [`meta-llama/Meta-Llama-3.1-70B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct)，并应用 `UltraFeedback` 任务，根据不同维度（如有用性、诚实性、遵循指令的能力、真实性）来评判响应。对于 SFT 数据集，您可以改用 [`PrometheusEval`](../papers/prometheus.md)。\n",
    "\n",
    "- 组件：使用 `InferenceEndpointsLLM` 的 `UltraFeedback` 任务\n",
    "- 输入列：`instruction`、`generations`\n",
    "- 输出列：`ratings`、`rationales`、`distilabel_metadata`、`model_name`\n",
    "\n",
    "根据你的使用场景并为了提高结果，你可以使用任何 [你选择的其他 LLM](https://distilabel.argilla.io/latest/components-gallery/llms/)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'instruction': \"What's the capital of Spain?\",\n",
       "  'generations': ['Madrid', 'Barcelona'],\n",
       "  'ratings': [5, 1],\n",
       "  'rationales': [\"The answer is correct, directly addressing the question, and is free of hallucinations or unnecessary details. It confidently provides the accurate information, aligning perfectly with the user's intent.\",\n",
       "   \"The answer is incorrect as Barcelona is not the capital of Spain. This introduces a significant inaccuracy, failing to provide helpful information and deviating entirely from the user's intent.\"],\n",
       "  'distilabel_metadata': {'raw_output_ultra_feedback_0': \"#### Output for Text 1\\nRating: 5 (Excellent)\\nRationale: The answer is correct, directly addressing the question, and is free of hallucinations or unnecessary details. It confidently provides the accurate information, aligning perfectly with the user's intent.\\n\\n#### Output for Text 2\\nRating: 1 (Low Quality)\\nRationale: The answer is incorrect as Barcelona is not the capital of Spain. This introduces a significant inaccuracy, failing to provide helpful information and deviating entirely from the user's intent.\"},\n",
       "  'model_name': 'meta-llama/Meta-Llama-3.1-70B-Instruct'}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate_responses = UltraFeedback(\n",
    "    aspect=\"overall-rating\",\n",
    "    llm=InferenceEndpointsLLM(\n",
    "        model_id=\"meta-llama/Meta-Llama-3.1-70B-Instruct\",\n",
    "        tokenizer_id=\"meta-llama/Meta-Llama-3.1-70B-Instruct\",\n",
    "        generation_kwargs={\"max_new_tokens\": 512, \"temperature\": 0.7},\n",
    "    ),\n",
    "    pipeline=Pipeline(name=\"showcase-pipeline\"),\n",
    ")\n",
    "evaluate_responses.load()\n",
    "next(\n",
    "    evaluate_responses.process(\n",
    "        [\n",
    "            {\n",
    "                \"instruction\": \"What's the capital of Spain?\",\n",
    "                \"generations\": [\"Madrid\", \"Barcelona\"],\n",
    "            }\n",
    "        ]\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 仅保留必要的列\n",
    "\n",
    "我们将去除不需要的列。\n",
    "\n",
    "- 组件：`KeepColumns`\n",
    "- 输入列：`system`、`instruction`、`chosen`、`rejected`、`generations`、`ratings`、`rationales`、`distilabel_metadata` 和 `model_name`\n",
    "- 输出列：`instruction`、`chosen`、`rejected`、`generations` 和 `order`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'instruction': \"What's the capital of Spain?\",\n",
       "  'generations': ['Madrid', 'Barcelona'],\n",
       "  'order': ['chosen', 'rejected'],\n",
       "  'ratings': [5, 1],\n",
       "  'rationales': ['', ''],\n",
       "  'model_name': 'meta-llama/Meta-Llama-3.1-70B-Instruct'}]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keep_columns = KeepColumns(\n",
    "    columns=[\n",
    "        \"instruction\",\n",
    "        \"generations\",\n",
    "        \"order\",\n",
    "        \"ratings\",\n",
    "        \"rationales\",\n",
    "        \"model_name\",\n",
    "    ],\n",
    "    pipeline=Pipeline(name=\"showcase-pipeline\"),\n",
    ")\n",
    "keep_columns.load()\n",
    "next(\n",
    "    keep_columns.process(\n",
    "        [\n",
    "            {\n",
    "                \"system\": \"\",\n",
    "                \"instruction\": \"What's the capital of Spain?\",\n",
    "                \"chosen\": \"Madrid\",\n",
    "                \"rejected\": \"Barcelona\",\n",
    "                \"generations\": [\"Madrid\", \"Barcelona\"],\n",
    "                \"order\": [\"chosen\", \"rejected\"],\n",
    "                \"ratings\": [5, 1],\n",
    "                \"rationales\": [\"\", \"\"],\n",
    "                \"model_name\": \"meta-llama/Meta-Llama-3.1-70B-Instruct\",\n",
    "            }\n",
    "        ]\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### （可选）进一步的数据整理\n",
    "\n",
    "你可以使用 Argilla 进一步整理您的数据。\n",
    "\n",
    "- 组件：`PreferenceToArgilla` 步骤\n",
    "- 输入列：`instruction`、`generations`、`generation_models`、`ratings`\n",
    "- 输出列：`instruction`、`generations`、`generation_models`、`ratings`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "to_argilla = PreferenceToArgilla(\n",
    "    dataset_name=\"cleaned-dataset\",\n",
    "    dataset_workspace=\"argilla\",\n",
    "    api_url=\"https://[your-owner-name]-[your-space-name].hf.space\",\n",
    "    api_key=\"[your-api-key]\",\n",
    "    num_generations=2\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 运行管道"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面，你可以看到完整管道定义:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "with Pipeline(name=\"clean-dataset\") as pipeline:\n",
    "\n",
    "    load_dataset = LoadDataFromDicts(\n",
    "        data=dataset, output_mappings={\"question\": \"instruction\"}\n",
    "    )\n",
    "\n",
    "    evaluate_responses = UltraFeedback(\n",
    "        aspect=\"overall-rating\",\n",
    "        llm=InferenceEndpointsLLM(\n",
    "            model_id=\"meta-llama/Meta-Llama-3.1-70B-Instruct\",\n",
    "            tokenizer_id=\"meta-llama/Meta-Llama-3.1-70B-Instruct\",\n",
    "            generation_kwargs={\"max_new_tokens\": 512, \"temperature\": 0.7},\n",
    "        ),\n",
    "    )\n",
    "\n",
    "    keep_columns = KeepColumns(\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"generations\",\n",
    "            \"order\",\n",
    "            \"ratings\",\n",
    "            \"rationales\",\n",
    "            \"model_name\",\n",
    "        ]\n",
    "    )\n",
    "\n",
    "    to_argilla = PreferenceToArgilla(\n",
    "        dataset_name=\"cleaned-dataset\",\n",
    "        dataset_workspace=\"argilla\",\n",
    "        api_url=\"https://[your-owner-name]-[your-space-name].hf.space\",\n",
    "        api_key=\"[your-api-key]\",\n",
    "        num_generations=2,\n",
    "    )\n",
    "\n",
    "    load_dataset.connect(evaluate_responses)\n",
    "    evaluate_responses.connect(keep_columns)\n",
    "    keep_columns.connect(to_argilla)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们来运行管道，清理我们的偏好数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "distiset = pipeline.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们检查一下！如果你已经将数据加载到 Argilla 中，你可以在 [Argilla UI 中开始标注](https://docs.argilla.io/latest/how_to_guides/annotate/)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以将数据集推送到 Hub 以便与社区共享，并 [嵌入它以探索数据](https://huggingface.co/docs/hub/datasets-viewer-embed)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "distiset.push_to_hub(\"[your-owner-name]/example-cleaned-preference-dataset\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<iframe\n",
    "  src=\"https://huggingface.co/datasets/distilabel-internal-testing/example-cleaned-preference-dataset/embed/viewer/default/train\"\n",
    "  frameborder=\"0\"\n",
    "  width=\"100%\"\n",
    "  height=\"560px\"\n",
    "></iframe>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在本教程中，我们展示了使用 distilabel 构建清理偏好数据集管道的详细步骤。然而，你可以根据自己的使用场景自定义此管道，例如清理 SFT 数据集或添加自定义步骤。\n",
    "\n",
    "我们以一个偏好数据集作为起点，并通过打乱数据来避免任何偏差。接下来，我们使用一个模型通过无服务器的 Hugging Face 推理 API 评估了响应，遵循了 UltraFeedback 标准。最后，我们保留了必要的列，并使用 Argilla 进行了进一步的数据整理。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "distilabel-tutorials",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
